Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use relative file paths to call linters #1877

Merged
merged 15 commits into from
Apr 19, 2023
Merged

Conversation

nvuillam
Copy link
Member

Fixes #1875

@nvuillam nvuillam force-pushed the features/relative-files branch from cae0336 to 62d1ae5 Compare September 15, 2022 21:59
@nvuillam
Copy link
Member Author

nvuillam commented Sep 15, 2022

🦙 MegaLinter status: ⚠️ WARNING

Descriptor Linter Files Fixed Errors Elapsed time
✅ BASH bash-exec 6 0 0.02s
✅ BASH shellcheck 6 0 0.17s
✅ BASH shfmt 6 0 0 0.45s
✅ COPYPASTE jscpd yes no 3.93s
✅ DOCKERFILE hadolint 117 0 24.37s
✅ JSON eslint-plugin-jsonc 23 0 0 3.13s
✅ JSON jsonlint 21 0 0.27s
✅ JSON v8r 23 0 18.03s
⚠️ MARKDOWN markdownlint 313 0 232 8.8s
✅ MARKDOWN markdown-link-check 313 0 6.76s
✅ MARKDOWN markdown-table-formatter 313 0 0 24.79s
✅ OPENAPI spectral 1 0 1.86s
⚠️ PYTHON bandit 188 57 2.77s
✅ PYTHON black 188 0 0 6.03s
✅ PYTHON flake8 188 0 2.41s
✅ PYTHON isort 188 0 0 1.02s
✅ PYTHON mypy 188 0 10.68s
✅ PYTHON pylint 188 0 17.23s
⚠️ PYTHON pyright 188 251 21.73s
✅ PYTHON ruff 188 0 0 0.6s
✅ REPOSITORY checkov yes no 44.77s
✅ REPOSITORY git_diff yes no 0.47s
✅ REPOSITORY secretlint yes no 23.02s
✅ REPOSITORY trivy yes no 36.57s
✅ SPELL cspell 629 0 34.76s
⚠️ SPELL vale 222 59 48.89s
✅ XML xmllint 3 0 0 0.45s
✅ YAML prettier 157 0 0 6.68s
✅ YAML v8r 99 0 178.99s
✅ YAML yamllint 158 0 1.97s

See detailed report in MegaLinter reports

MegaLinter is graciously provided by OX Security

File filtering management with relative paths


Fix workspace


Manage relative paths of files in sub directories


mypy fix


Manage file_sub_directory with relative files


[MegaLinter] Apply linters fixes

Temporary increase timeout of build dev job


Fix filter test classes


Add test case file


temp disable test sarif output

quick build
flake8 fix


[MegaLinter] Apply linters fixes
@nvuillam nvuillam force-pushed the features/relative-files branch from 216dbc0 to b3ef6ab Compare November 21, 2022 23:30
@bdovaz
Copy link
Collaborator

bdovaz commented Dec 8, 2022

How is it going? What is blocking it?

@nvuillam
Copy link
Member Author

@bdovaz memory crash within the docker image... really hard to debug :/
but you can try with temp docker image generated by CI job

@nvuillam
Copy link
Member Author

image

:/

@nvuillam
Copy link
Member Author

nvuillam commented Dec 22, 2022

🦙 MegaLinter status: ⚠️ WARNING

Descriptor Linter Files Fixed Errors Elapsed time
✅ BASH bash-exec 6 0 0.01s
✅ BASH shellcheck 6 0 0.13s
✅ BASH shfmt 6 0 0 0.04s
✅ COPYPASTE jscpd yes no 2.63s
✅ DOCKERFILE hadolint 117 0 14.49s
✅ JSON eslint-plugin-jsonc 23 0 0 1.67s
✅ JSON jsonlint 21 0 0.18s
✅ JSON npm-package-json-lint yes no 0.6s
✅ JSON v8r 23 0 15.49s
⚠️ MARKDOWN markdownlint 313 2 232 5.66s
✅ MARKDOWN markdown-link-check 313 0 5.36s
✅ MARKDOWN markdown-table-formatter 313 2 0 17.2s
✅ OPENAPI spectral 1 0 1.26s
⚠️ PYTHON bandit 188 57 2.07s
✅ PYTHON black 188 0 0 3.75s
✅ PYTHON flake8 188 0 1.8s
✅ PYTHON isort 188 0 0 0.45s
✅ PYTHON mypy 188 0 7.6s
✅ PYTHON pylint 188 0 11.27s
⚠️ PYTHON pyright 188 251 15.13s
✅ PYTHON ruff 188 0 0 0.1s
✅ REPOSITORY checkov yes no 30.76s
⚠️ REPOSITORY devskim yes 1145 4.93s
✅ REPOSITORY dustilock yes no 1.97s
✅ REPOSITORY git_diff yes no 0.04s
✅ REPOSITORY secretlint yes no 12.24s
✅ REPOSITORY syft yes no 1.11s
✅ REPOSITORY trivy yes no 24.51s
✅ SPELL cspell 629 0 21.27s
⚠️ SPELL vale 222 59 34.42s
✅ XML xmllint 3 0 0 0.03s
✅ YAML prettier 157 0 0 4.11s
✅ YAML v8r 99 0 125.75s
✅ YAML yamllint 158 0 1.58s

See detailed report in MegaLinter reports

You could have same capabilities but better runtime performances if you request a new MegaLinter flavor.

MegaLinter is graciously provided by OX Security

@github-actions
Copy link
Contributor

This pull request has been automatically marked as stale because it has not had recent activity.
It will be closed in 14 days if no further activity occurs.
Thank you for your contributions.

If you think this pull request should stay open, please remove the O: stale 🤖 label or comment on the pull request.

@github-actions github-actions bot added the O: stale 🤖 This issue or pull request is stale, it will be closed if there is no activity label Jan 22, 2023
@bdovaz bdovaz removed the O: stale 🤖 This issue or pull request is stale, it will be closed if there is no activity label Jan 22, 2023
@github-actions
Copy link
Contributor

This pull request has been automatically marked as stale because it has not had recent activity.
It will be closed in 14 days if no further activity occurs.
Thank you for your contributions.

If you think this pull request should stay open, please remove the O: stale 🤖 label or comment on the pull request.

@github-actions github-actions bot added the O: stale 🤖 This issue or pull request is stale, it will be closed if there is no activity label Feb 22, 2023
@bdovaz bdovaz added nostale This issue or pull request is not stale, keep it open and removed O: stale 🤖 This issue or pull request is stale, it will be closed if there is no activity labels Feb 22, 2023
@bdovaz
Copy link
Collaborator

bdovaz commented Mar 26, 2023

@nvuillam can you make a rebase? To at least check from time to time if this problem has been solved.

I would do it myself but I have doubts with utils.py and I prefer you to solve it.

@nvuillam
Copy link
Member Author

@bdovaz done :)

@bdovaz
Copy link
Collaborator

bdovaz commented Apr 12, 2023

@nvuillam Is it just me or seeing the result of the jobs, the random error that occurred no longer exists?

@bdovaz
Copy link
Collaborator

bdovaz commented Apr 18, 2023

@nvuillam Is it just me or seeing the result of the jobs, the random error that occurred no longer exists?

ping @nvuillam

@nvuillam
Copy link
Member Author

The error was not random... it was always and hard to diagnose its source ^^

@nvuillam
Copy link
Member Author

Not crashing indeed !
But did i rebuild?

@nvuillam
Copy link
Member Author

nvuillam commented Apr 18, 2023

/build

Command run output
Build command workflow started.
Installing dependencies
Running script ./build.sh
Build command workflow completed without updating files.

@nvuillam
Copy link
Member Author

@bdovaz I have a good feeling about this one... :D :D

@nvuillam nvuillam merged commit 4d85759 into main Apr 19, 2023
@nvuillam nvuillam deleted the features/relative-files branch April 19, 2023 21:35
@nvuillam
Copy link
Member Author

@bdovaz @echoix @Kurt-von-Laven

Now I'm kind a afraid to release a new version... this PR is potentially very impacting, if you have some time please use beta versions in your repos to make sure that it still works well with relative files :) ( I'll obviously do the same on my side ^^ )

@echoix
Copy link
Collaborator

echoix commented Apr 19, 2023

You could release a RC in the weekend and look at the number of pulls the following week to make sure that there was enough pulls.

@Kurt-von-Laven
Copy link
Collaborator

Thanks for the heads up, @nvuillam. I tried the documentation, dotnet, javascript, and python flavors of the beta version on all of our repositories and encountered three issues. I am listing them all here for now since the second and third seem like they could potentially be related, but I am happy to file issues about these when I have more time tomorrow. The third issue is a preexisting issue that was made more visible by #2455, because previously ESLint silently didn't run at all locally.

  1. CSharpier is broken; I am curious whether anyone else encounters this issue:

    ❌ Linted [CSHARP] files with [csharpier]: Found 1 error(s) - (1.18s)
    --Error detail:
    Run "dotnet tool restore" to make the "dotnet-csharpier" command available.
    
    Unable to get number of errors with regex_number and Issues found: ([0-9]+) in .* files
    
  2. CSpell is broken when importing dictionaries that aren't bundled:

    ❌ Linted [SPELL] files with [cspell]: Found 1 error(s) - (18.84s)
    --Error detail:
    Configuration Error: Failed to read config file: "/tmp/lint/@cspell/dict-medicalterms/cspell-ext.json"
    CSpell: Files checked: 0, Issues found: 0 in 0 files
    
    ❌ Error(s) have been found during linting
    

    Our cspell.config.yaml contains:

    import:
      - "@cspell/dict-medicalterms/cspell-ext.json"

    Our .mega-linter.yaml contains:

    SPELL_CSPELL_CONFIG_FILE: LINTER_DEFAULT
    SPELL_CSPELL_PRE_COMMANDS:
      - command: npm install @cspell/dict-medicalterms@3.0.0
  3. In our Yarn TypeScript projects, there is one error per file of the following form:

    ❌ Linted [TYPESCRIPT] files with [eslint]
    --Error detail:
    
    /tmp/lint/some_file.ts
      0:0  error  Parsing error: File '@tsconfig/node18-strictest-esm/tsconfig.json' not found
    

    Our tsconfig.json contains:

    {
      "extends": "@tsconfig/node18-strictest-esm/tsconfig.json"
    }

    Our package.json contains:

    {
      "devDependencies": {
        "@tsconfig/node18-strictest-esm": "1.0.1"
      }
    }

    Our .eslintrc.yaml is:

    root: true
    extends:
      - eslint:recommended
      - plugin:@typescript-eslint/recommended
      - prettier
    parser: "@typescript-eslint/parser"
    parserOptions:
      project:
        - tsconfig.json
    plugins:
      - "@typescript-eslint"
    env:
      node: true

@Kurt-von-Laven
Copy link
Collaborator

If it makes you feel any better, passing relative file paths fixed the error originally reported in #1572, getting ESLint running in list_of_files mode for npm projects and unblocking running ESLint in list_of_files mode for Yarn PnP projects.

@nvuillam
Copy link
Member Author

@Kurt-von-Laven I'm trying to fix that in #2601 ... the idea is the following:

  • check if PRE_COMMANDS contain "npm" or "yarn"
  • if yes:
    • copy /node_deps into workspace+"node_modules"
    • Add workspace+"node_modules" in PATH
    • run the PRE_COMMANDS

Do you think it could work ? ^^

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
nostale This issue or pull request is not stale, keep it open
Projects
None yet
Development

Successfully merging this pull request may close these issues.

v8r failing due to megalinter not executing within project path
4 participants